Domain Adaptation with Pre-trained Transformers for Query-Focused Abstractive Text Summarization

Abstract The Query-Focused Text Summarization (QFTS) task aims at building systems that generate the summary of text document(s) based on given query. A key challenge in addressing this is lack large labeled data for training summarization model. In article, we address by exploring a series domain adaptation techniques. Given recent success pre-trained transformer models wide range natural language processing tasks, utilize such to abstractive summaries QFTS both single-document and multi-document scenarios. For adaptation, apply variety techniques using transformer-based including transfer learning, weakly supervised distant supervision. Extensive experiments six datasets show our proposed approach very effective generating while setting new state-of-the-art result several across set automatic human evaluation metrics.